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ABSTRACT 

An effective test and measurement course in 
psychology should expose students to a variety of available 
psychological tests, as well as to the mechanics of test construction 
and evaluation. In a test and measurement course at the State 
University of New York's College at Cortland, the course is divided 
into two components with an overlaying group project. In the first 
component, students are introduced to fundamental concepts of test 
design, construction, and evaluation. In the second component, 
students use these fundamental concepts to evaluate the classic tests 
in each of the areas represented in psychological measurements. For 
the group project, students form groups of five members, with each 
member becoming an "expert" in one of the upper levels of Bloom's 
taxonomy of cognitive domains (i.e., comprehension, application, 
analysis, synthesis, and evaluation). Each group develops a test 
consisting of one question each from the five levels and critically 
review other groups' tests. The new measure is then validated and 
administered to a group of subjects along with a second external 
measure. The approach effectively reinforces the fundamentals of test 
construction and evaluation, introducing students to one measurement 
domain in great depth and provides students with the opportunity to 
work in interrelating, collaborative groups. An organizational chart 
of the test-creation project is appended. (TGI) 
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Balancing coverage of the range of core topics with in-depth coverage of selected issues is a 
major concern in designing a Test and Measurement course. Many courses and texts in this 
area look like lists of tests in print. Students are presented with descriptions and examples 
of literally hundreds of tests designed to evaluate intelligence, personality, academic 
aptitudes, vocational preferences and a host of other individual dimensions. At the other 
extreme some texts bombard students with advanced statistical formulas, seeming to imply 
that a course in Test and Measurements is nothing but an applied statistics course. I feel 
that it is important to integrate these two approaches so that students gain an appreciation 
of the variety of available psychological tests, and an understanding of the mechanics of test 
construction and evaluation. 

My Test and Measurement course is divided into two content components with an 
overlaying group project. In the first phase of the course I introduce fundamental concepts 
of test design, construction and evaluation. Students read about, and discuss in class, issues 
related to item generation, standardization of administration and scoring procedures, 
establishment of norms, the appropriate use of the various measurement scales, factor 
analysis, item analysis, and methods for determining reliability and validity. In the second 
phase students use these fundamental concepts to evaluate the classic tests in each of the 
areas represented in psychological measurements. 

I feel that this progression from the general to the specific provides a sound initial 
introduction to the domain of test and measurements. However, I want my students to 
more thoroughly appreciate the mechanics of test construction and I believe the best way to 
do that is to have them actually create a test I use depth of cognitive processing as the 
focus of this exercise. Students are introduced to the wealth of theories and associated 
measures that exist in this domain. They are then set the task of developing a test that 
operationalizes Bloom’s Taxonomy of the Cognitive Domain (1956). 



In order to develop the Bloom measure students work in "core groups" of five. Each 
member of the group selects one of the upper levels of the taxonomy (comprehension, 
application, analysis, synthesis and evaluation) that they wish to become an "expert" in. 
These individuals from each of the "core groups" (usually five per class) form "expert 
groups" to study their specific level. This means that each student is actually a member of 
two working groups simultaneously, the "core group" and the "expert group". Individual 
students are responsible for creating one set of questions (usually a paragraph situation 
with five objective questions) that represents their level of expertise. The "core group" then 
works together to produce a cohesive test consisting of five questions, one representing 
each level. These tests are then exchanged with other "core groups" for critical review. 
Groups determine if the questions they are reviewing represent the level of the taxonomy 
they were designed to measure, if they are bias free and conform to the requirements of 
sound test construction discussed in class. At this point each group selects the three 
questions per level that they feel are best constructed. (See attached figure for a graphic 
representation of the entire process.) 

After the test is developed each "core group" selects one of the components of validity as 
outlined by Messick (1995) that they wish to use to validate their new measure. In order to 
accomplish this one of the groups develops a thorough description of the construct of depth 
of cognitive performance. A second group develops a model of the predicted scoring 
patterns based on Bloom’s theory. A third group determines appropriate alternative 
measures to use to establish the external aspects of validity. A fourth group examines the 
issues surrounding generalizability of the new measure. The fifth group focuses on the 
possible consequences of the use of the new measure. 



The new Bloom test is then administered to a group of subjects along with a second 
(external) measure of learning style. The results of the two tests are compared to 
determine criterion validity. The results of the Bloom test are factor analyzed to ascertain 
whether or not the predicted factor structure exists. The results are also subjected to item 
analysis to examine the power of each item. 



This project design accomplishes several of my goals. It reinforces the fundamentals of test 
construction and evaluation and introduces students to one measurement domain in great 
depth. This project also introduces the students to working in interrelating collaborative 
groups while reinforcing the content components of the course. Each member of each 
group must contribute a critical component of the group project. This model allows for 
individual and group accountability and assessment of performance. Finally, to ensure that 
each group member is familiar with all aspects of the project, questions concerning the 
development process, the theoretical underpinnings, and the interpretation of the specific 
results of the tests form the basis of the final exam in the course. 
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